Embodiments herein generally relate to systems that provide an interface to users to allow the users to locate and select colors of items and more particularly to systems and methods that allow users to input natural language commands to perform color selection.
Color sets or collections are in widespread use in a range of commercial and consumer industries. Color sets or collections may be the names of actual colored objects in the collection, as is the case for a box of crayons or a collection of house paints, or they may be simply a collection of names associated with defined color specifications, as is the case for Pantone and other color collections used in the graphic design industry. In the case of Pantone and similar color collections, the collection can be implemented either in software or as books of colored sample swatches. The software implementation is essentially a file that relates Pantone names to color specifications. In most cases the colors in the collection are assigned names that may be more or less descriptive of the appearance of the color. One commercially available color selection system, Pantone (available from Pantone, Inc. Carlstadt, N.J., USA) names its colors using names like Pantone 172 CVC, which is a vivid orange color. Clearly finding the correct Pantone color name to match an intended color can be difficult.
Pantone attempts to assist users by producing swatch books in which the patches are grouped into similar colors and arranged in something like a spectral progression. Vendors such as X-Rite (Grand Rapids, Mich., USA) offer a tool which returns the closest Pantone color to a given color on the screen. Similar problems arise with paint colors or crayon colors where the names used often convey little or no information about the appearance of the color. Paint manufacturers try to address this problem by arrangement of colors on a color card according to a color order scheme.
Natural language presents a number of unique opportunities for assisting this named color selection problem. These opportunities are the subject of the embodiments of this disclosure. More specifically, the embodiments herein comprise a color selector that allows users to find colors from a collection or set of available colors by providing a natural language description of the color appearance desired. This is particularly useful in situations where the colors in the collection have names that convey little or no information concerning the appearance of the color.
In a simple embodiment, the user enters a verbal description of the desired color appearance and the application returns a set of color patches, displayed on the monitor, together with the names of the colors. A user then uses language commands to adjust the color and range of colors selected. In addition to natural language commands entered through a microphone or keyboard, the embodiments herein also allow the use of a pointing device and other user interface designs in combination with the natural language interface.
Thus, one method embodiment herein starts by receiving initial user input. This initial user input can comprise initial natural language commands that identify an initial color selection. The method displays a plurality of initial color samples or patches that correspond to the initial color selection in a two- or three-dimensional grid (e.g., using a pseudo 3D plot—using perspective to give the illusion of 3D) on a graphic user interface, such as a computer monitor. The axes of the two-dimensional grid can correspond to a predetermined standard axis system. Also, simultaneously displayed on the graphic user interface are assigned names of the initial color regions, and these names are positioned adjacent (above, below, beside, in, on, etc.) the initial color regions. This displays the names of the colors as they are defined in the collection being searched. If it is a Pantone collection then Pantone's names are displayed. If it was a collection of Crayon colors then it would be Crayon's names that are displayed. However, many times the initial natural language commands are different than assigned names (e.g., Pantone names) of the initial color regions because color names assignments are often made based on criteria that are fundamentally different than names commonly given by casual users e.g. for marketing purposes.
After displaying the initial color regions, the method may (or may not) receive at least one additional user input. Such additional user inputs can comprise additional natural language commands and/or a revised axis selection. These additional natural language commands can comprise a color change magnitude, a color change direction, and a color change property.
For example, a natural language command of “slightly more blue” spoken or typed into a graphic user interface provides the color change magnitude (“slightly”), color change direction (“more”), and color change property (“blue”) used by embodiments herein to revise the colors displayed to the user.
Thus, the method revises the initial color selection to a revised color selection based on the color change magnitude, the color change direction, and the color change property provided in natural language format by the user. Then, the method can display revised color regions (patches) that correspond to the revised color selection in the two- or three dimensional grid of the display. However, the axes of the two-or three dimensional grids can also be modified to correspond to the revised axis selection provided by the user in the additional input. The selection of the axes will affect the spatial relationships of the colors to each other as they are displayed on the user interface. For example, if the axes are chosen to be lightness on the vertical axis and hue on the horizontal axis then color samples or patches will be arranged vertically in order of their lightness and horizontally in order of their hue.
In a similar manner to the display of the initial color regions, the display can also simultaneously display the assigned names of the revised color samples adjacent (above, below, beside, in, on, etc.) the revised color samples. The system may be configured such that the names are not displayed unless a pointing device is placed over the color or clicked on the color in order to avoid the display becoming too cluttered. Again, the additional natural language commands may be different than assigned names of the revised color samples. Indeed, the additional language commands comprise color change magnitudes, color change directions, and color change properties and do not necessarily correspond to the names a specific computer program may assign to colors.
The color change property comprises an indication of color, luminance, saturation, etc. The color change magnitude comprises an indication of how much of the color change property should be applied to the initial color selection. The color change direction comprises an indication of whether the color change property should be increased or decreased in the initial color selection.
The initial natural language commands comprise words maintained within at least one previously established library of natural language color names. Similarly, the additional natural language commands comprises a plurality of words maintained within at least one previously established library of natural language color modifiers (which can be the same library or different pre-established libraries).
When displaying the initial color samples and the revised color samples, the method displays a principal color patch (corresponding respectively to the best matching sample to either the initial color selection or the revised color selection) and displays secondary color samples having similarities to the principal color patch. Ones of the secondary color samples that are more similar to the principal color patch are positioned closer to the principal color patch, and ones of the secondary color samples that are less similar to the principal color patch are positioned further from the principal color patch. The concept of similarity depends on the axes and measurement system you are using. In Figure one, the invention has defined axes to be lightness and colorfulness. Thus, similarity is measured in these terms. If one changed the axes properties to be, say, hue and saturation, then the similarity relationships might well change.
Another method embodiment similarly begins by receiving initial user input comprising such natural language commands that identify the initial color selection. However, this embodiment then performs a step of matching the initial color selection to computer program colors used by an associated computer program to produce initial matching colors. Therefore, this embodiment is useful as an intermediary program that provides a service of interfacing between the user and the associated computer program that requires color selection.
This method similarly displays the initial color regions corresponding to the initial color selection in a two-dimensional grid and receives additional user input comprising additional natural language commands and a revised axis selection. This embodiment also revises the initial color selection to a revised color selection based on the color change magnitude, the color change direction, and the color change property. Then this method matches the revised color selection to the computer program colors to produce revised matching colors. These refined revised matching colors are displayed as revised color regions in the two-dimensional grid. Again, the axes of the two-dimensional grid can correspond to the revised axis selection and the assigned names can also be displayed on the graphic user interface.
Further, this embodiment can receive a selection of one or more of the revised color samples as one or more selected colors from the user. The selected color(s) are then provided to the associated computer program.
When matching of the initial user input to the computer program colors, this embodiment ignores the assigned names of the colors. To the contrary, rather than trying to match the natural language commands to arcane or irrelevant names given to the colors by the associated computer program, the embodiments herein match colorimetric or color appearance characteristic of the initial color selection and the revised color selection to the colorimetric or color appearance characteristics of the computer program colors (in a colorspace) to determine which of the computer program color patches to display to the user.
In other words, the embodiments herein use the pre-established libraries to identify the color (in terms of colorspace, colorimetric, or spectral characteristics) that is being requested by the user through the user's natural language commands. Then, the embodiments match these colorimetric or spectral characteristics (or colorspace) to the most similar color available in the associated computer program with which the embodiments herein are operating, without relying upon the names assigned to the available colors by the associated computer program. Thus, the embodiments herein convert the user's words to a color and then match that color to the associated computer program color, without performing any name matching. Then, when the embodiments herein display the available colors, the user is provided with the names of those colors as defined by the associated computer program. By operating in this manner, the user does not need to know the names of the colors that the associated computer program has assigned to colors, but instead the user can rely upon casual (generic) natural language names and commands to locate their desired color.
These and other features are described in, or are apparent from, the following detailed description.